Issue 16277 #16716

Nekmo · 2018-06-12T21:51:21Z

At least skimmed through adding new extractor tutorial and youtube-dl coding conventions sections
Searched the bugtracker for similar pull requests
Checked the code with flake8
I am the original author of this code and I am willing to release it under Unlicense
Bug fix

Fixed Issue #16277: Atresplayer broken: ERROR: Unsupported URL

Fix error 500

Fixed code typo

Nekmo · 2018-06-12T22:15:09Z

Fixed Python2 support :( (urllib.error)

Nekmo · 2018-06-13T19:46:26Z

Please check again "do-not-merge" label cc/ @dstftw

Aleixenandros · 2018-06-20T07:00:05Z

¿Por qué no lo implementan en las nuevas versiones?

Funciona perfecto desde el repositorio de @Nekmo

dstftw · 2018-06-22T20:57:37Z

youtube_dl/extractor/atresplayer.py

+try:
+    from urllib.error import HTTPError
+except ImportError:
+    from urllib2 import HTTPError


compat_HTTPError.

dstftw · 2018-06-22T20:58:00Z

youtube_dl/extractor/atresplayer.py


 class AtresPlayerIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/television/[^/]+/[^/]+/[^/]+/(?P<id>.+?)_\d+\.html'
+    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/[^/]+/[^/]+/' \
+                 r'[^/]+/[^/]+/[^/_]+_(?P<id>[A-z0-9]+)/?'


/? is pointless at the end.

dstftw · 2018-06-22T20:58:36Z

youtube_dl/extractor/atresplayer.py


 class AtresPlayerIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/television/[^/]+/[^/]+/[^/]+/(?P<id>.+?)_\d+\.html'
+    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/[^/]+/[^/]+/' \
+                 r'[^/]+/[^/]+/[^/_]+_(?P<id>[A-z0-9]+)/?'


Keep the regex on a single line.

dstftw · 2018-06-22T20:58:54Z

youtube_dl/extractor/atresplayer.py


 class AtresPlayerIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/television/[^/]+/[^/]+/[^/]+/(?P<id>.+?)_\d+\.html'
+    _VALID_URL = r'https?://(?:www\.)?atresplayer\.com/[^/]+/[^/]+/' \
+                 r'[^/]+/[^/]+/[^/_]+_(?P<id>[A-z0-9]+)/?'


A-z incorrect.

dstftw · 2018-06-22T20:59:13Z

youtube_dl/extractor/atresplayer.py

    _NETRC_MACHINE = 'atresplayer'
    _TESTS = [
        {
-            'url': 'http://www.atresplayer.com/television/programas/el-club-de-la-comedia/temporada-4/capitulo-10-especial-solidario-nochebuena_2014122100174.html',
-            'md5': 'efd56753cda1bb64df52a3074f62e38a',
+            'url': 'https://www.atresplayer.com/lasexta/programas/el-'


Don't split. Same everywhere.

dstftw · 2018-06-22T21:02:18Z

youtube_dl/extractor/atresplayer.py

        }

+        self._download_webpage(self._LOGIN_URL, None, 'get login page')
        request = sanitized_Request(


Inline into actual _download_* call.

I don't understand this comment.

@Nekmo You can make json POST with _download_json, and set expected statuses

for POST use data=urlencode_postdata(form_data)

From common.py

def _download_json( self, url_or_request, video_id, note='Downloading JSON metadata', errnote='Unable to download JSON metadata', transform_source=None, fatal=True, encoding=None, data=None, headers={}, query={}, expected_status=None): """ Return the JSON object as a dict. See _download_webpage docstring for arguments specification. """

The response from the server is not a json. This request is to set cookies and session.

dstftw · 2018-06-22T21:02:29Z

youtube_dl/extractor/atresplayer.py

+        except JSONDecodeError:
+            return original_exception
+        if isinstance(data, dict) and 'error' in data:
+            return ExtractorError('{} returned error: {} ({})'.format(


dstftw · 2018-06-22T21:02:52Z

youtube_dl/extractor/atresplayer.py

+            raise self._atres_player_error(e.exc_info[1].file.read(), e)
+
+        for source in video_data['sources']:
+            if source['type'] == "application/dash+xml":


Single quotes.

dstftw · 2018-06-22T21:03:04Z

youtube_dl/extractor/atresplayer.py

+            raise self._atres_player_error(e.exc_info[1].file.read(), e)
+
+        for source in video_data['sources']:
+            if source['type'] == "application/dash+xml":


Should not break if no type.

eg find type by URL ext

Already fixed on PR bbb857c

dstftw · 2018-06-22T21:03:30Z

youtube_dl/extractor/atresplayer.py

-            'thumbnail': thumbnail,
-            'duration': duration,
+            'title': video_data['titulo'],
+            'description': video_data['descripcion'],


Read coding conventions on optional/mandatory meta fields.

pereorga · 2019-04-08T20:48:02Z

FWIW, this is working fine locally, on both Windows 10 (Python 3.7.3) and Debian stable (Python 3.5.3).

It does not work, however, on an Ubuntu 16.04 instance that I have in DigitalOcean (Python 3.5.2). I guess it may be because of IP address-based geo-blocking? I am getting an HTTP 403:

~/dev/youtube-dl/youtube_dl$ python3 ./__main__.py -uUSERNAME -pPASSWORD https://www.atresplayer.com/lasexta/programas/salvados/temporada-14/francisco_5c9f49237ed1a885b9056c1f/

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-uUSERNAME', '-pPASSWORD', 'https://www.atresplayer.com/lasexta/programas/salvados/temporada-14/francisco_5c9f49237ed1a885b9056c1f/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.06.11
[debug] Git HEAD: bbb857c
[debug] Python version 3.5.2 (CPython) - Linux-4.4.0-143-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: none
[debug] Proxy map: {}
[AtresPlayer] get login page
[AtresPlayer] post to login form
[AtresPlayer] Set login session
[AtresPlayer] 5c9f49237ed1a885b9056c1f: Downloading player JSON
[AtresPlayer] 5c9f49237ed1a885b9056c1f: Downloading video JSON
Traceback (most recent call last):
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 579, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/netol/dev/youtube-dl/youtube_dl/YoutubeDL.py", line 2211, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/atresplayer.py", line 109, in _real_extract
    fatal=True)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 767, in _download_json
    data=data, headers=headers, query=query)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 752, in _download_json_handle
    encoding=encoding, data=data, headers=headers, query=query)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 599, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 588, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/home/netol/dev/youtube-dl/youtube_dl/__init__.py", line 472, in main
    _real_main(argv)
  File "/home/netol/dev/youtube-dl/youtube_dl/__init__.py", line 462, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/netol/dev/youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/home/netol/dev/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/common.py", line 500, in extract
    ie_result = self._real_extract(url)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/atresplayer.py", line 113, in _real_extract
    raise self._atres_player_error(e.exc_info[1].file.read(), e)
  File "/home/netol/dev/youtube-dl/youtube_dl/extractor/atresplayer.py", line 80, in _atres_player_error
    data = json.loads(body_response)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

Nekmo added 17 commits February 17, 2018 20:04

Fix error 500

39ed09b

Merge pull request #1 from Nekmo/atresplayer-fix-500

867811c

Fix error 500

Fixed code typo

8364a6e

Merge pull request #2 from Nekmo/atresplayer-typo

2a78780

Fixed code typo

Fix error 500

adf916e

Fixed code typo

fd8c111

Download video from new Atresplayer API.

6ad850f

Extra data and test.

adc4b9a

Removed old variables.

8daa5ce

Test using skip_download and refactor.

e88344b

Catch server errors and refactor imports.

c734b06

Atresplayer login.

1a981ad

Refactor.

72a6740

Merge remote-tracking branch 'origin/master'

d9b855e

Merge branch 'master' into issue-16277

bcc4698

Flake8

11ba372

Merge branch 'master' into issue-16277

d0909d8

dstftw added the do-not-merge label Jun 12, 2018

Nekmo added 2 commits June 13, 2018 00:12

Fixed Python2 support on Atresplayer.

e9986ce

Merge remote-tracking branch 'origin/issue-16277' into issue-16277

1211787

dstftw requested changes Jun 22, 2018

View reviewed changes

dstftw added the pending-fixes label Jun 22, 2018

Atresplayer PR changes.

bbb857c

Nekmo mentioned this pull request Dec 1, 2018

Atresplayer broken: ERROR: Unsupported URL #16277

Closed

5 tasks

dstftw force-pushed the master branch from d99bab0 to e118a87 Compare January 23, 2019 18:41

remitamine closed this in 6d394a6 Oct 16, 2019

meunierd referenced this pull request in meunierd/youtube-dl Feb 13, 2020

[atresplayer] fix extraction(closes #16277)(closes #16716)

d929922

pareronia referenced this pull request in pareronia/youtube-dl Jun 22, 2020

[atresplayer] fix extraction(closes #16277)(closes #16716)

1891fc9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 16277 #16716

Issue 16277 #16716

Nekmo commented Jun 12, 2018 •

edited

Loading

Nekmo commented Jun 12, 2018

Nekmo commented Jun 13, 2018

Aleixenandros commented Jun 20, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

Nekmo Jul 13, 2018 •

edited

Loading

bato3 Jul 30, 2018 •

edited

Loading

Nekmo Jul 30, 2018 •

edited

Loading

dstftw Jun 22, 2018

dstftw Jun 22, 2018

dstftw Jun 22, 2018

bato3 Jul 30, 2018

Nekmo Jul 30, 2018

dstftw Jun 22, 2018

pereorga commented Apr 8, 2019

Issue 16277 #16716

Issue 16277 #16716

Conversation

Nekmo commented Jun 12, 2018 • edited Loading

Nekmo commented Jun 12, 2018

Nekmo commented Jun 13, 2018

Aleixenandros commented Jun 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nekmo Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

bato3 Jul 30, 2018 • edited Loading

Choose a reason for hiding this comment

Nekmo Jul 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pereorga commented Apr 8, 2019

Nekmo commented Jun 12, 2018 •

edited

Loading

Nekmo Jul 13, 2018 •

edited

Loading

bato3 Jul 30, 2018 •

edited

Loading

Nekmo Jul 30, 2018 •

edited

Loading